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Executive Summary 


In January 2018, Polk County, Iowa began a testing period for use of the Public Safety 
Assessment (“PSA”), an evidence-based pretrial risk assessment instrument. In July 2018, Linn 
County, Iowa began a similar test. The Laura and John Arnold Foundation (“LJAF”) had 
previously funded the creation of the PSA to assist judges in deciding which arrestees should be 
released while awaiting case disposition, and, if release were ordered, what level of monitoring 
should be imposed. The PSA scoring system purports to band arrestees by risk of failure to 
appear, new criminal activity, and new violent criminal activity. Judges provided the PSA may 
choose to consult the scores and corresponding recommendation among several factors in 
detennining initial release. The PSA’s purpose is to assist jurisdictions in their efforts to 
minimize rates of pre-disposition incarceration, failure to appear, and new (violent) criminal 
activity. 

LJAF, Polk County, and Linn County requested that the Access to Justice (“A2J”) Lab at 
Harvard Law School evaluate the PSA’s effects. Working with stakeholders in both counties, the 
A2J Lab designed and initiated a randomized control trial evaluation in which the PSA would be 
available to district associate judges in some cases but not in others. A comparison of the 
outcomes in the PSA and no-PSA groups will provide gold-standard information on the PSA’s 
effects. When the PSA and the randomized evaluation launched, the A2J Lab warned that, 
because many criminal cases can take a year or more to reach disposition, and because a 
sufficient number of cases must reach disposition to allow statistical analysis, credible 
information on the PSA’s effects would not be available until three to four years after launch. At 
LJAF’s request, however, the A2J Lab agreed to produce a report by December 3, 2018, 
discussing the results for the small fraction of randomized cases that had reached resolution by 
mid-fall of the same year. 

As the A2J Lab predicted, credible information about the PSA’s effects in Polk and Linn 
Counties is not yet available. Too few cases have reached disposition, and those cases that have 
reached disposition are not representative of the two jurisdictions’ overall arrestee profiles. At 
this time, based on the small amount of infonnation available, the A2J Lab observes no credible 
evidence that PSA availability decreases or increases rates of incarceration, failure to appear, 
new criminal activity, or new violent criminal activity. Moreover, because the present analysis 
took place under conditions of haste, the A2J Lab cannot rule out the possibility of data, coding, 
or comprehension errors, despite measures taken to eliminate each such danger. 

The A2J Lab’s evaluation design contemplated two years of randomization followed by a 
two-year follow-up period, the latter included so that cases could reach disposition and to 
observe post-disposition recidivism rates. The A2J Lab recommends that the evaluation be 
pennitted to finish so that credible information on the PSA’s effects will be available to policy 
makers who decide whether its use should continue. 
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I. Introduction 


Polk County, Iowa launched the Public Safety Assessment (“PSA”) in January 2018 for 
use in post-arrest release/detain decisions at initial appearance. In July 2018, Linn County 
followed suit. The PSA is a pretrial risk assessment instrument developed by the Laura and John 
Arnold Foundation (“LJAF”) for use in making release/detain decisions in criminal cases (and 
recommending levels of monitoring if the decision is to release). The algorithm on which the 
PSA is based bands arrestees according to risks regarding both public safety (in terms of 
committing new criminal acts and new violent criminal acts) and failure to appear for future 
court hearings. LJAF, in coordination with state and local stakeholders in Iowa, contacted the 
Access to Justice (“A2J”) Lab at Harvard Law School to perform a randomized evaluation of the 
PSA in Polk and Linn Counties, with a randomization period of two years and a two-year follow¬ 
up period. The A2J Lab began collecting data as soon as the PSA launched in both counties. At 
LJAF’s request, the A2J Lab also agreed to produce the present report. At the time of LJAF’s 
request, the A2J Lab informed all concerned that the time period between launch of the PSA and 
this report would almost certainly be too short to allow substantive conclusions. Because 
criminal cases can take months, sometimes more than a year, to reach disposition, the report 
would be based on only a few months of cases, and many of those cases would still be ongoing at 
the time the A2J Lab collected relevant data. Those predictions have proven true. This document 
thus explains the ongoing efforts to conduct a credible evaluation of the PSA, reports what the 
data show thus far, explains why the data do not allow useful conclusions as yet, and discusses 
prospects for the future. We begin with the reasons why no useful conclusions can be drawn at 
this time. 

II. Limits of the Current Data 

A. Low Statistical Power 

The full research design the A2J Lab is currently pursuing contemplates a two-year 
period for running the field experiment, followed by two years of follow-up to allow cases to 
conclude and to observe recidivism rates. The A2J Lab anticipates producing an interim report 
shortly after the end of the third study year based on the first year’s worth of randomized cases, 
i.e., at a time when follow-up for those cases is complete. These timeframes were chosen for two 
reasons. First, the A2J Lab estimated that two years of randomization would provide the number 
of cases necessary to observe policy-relevant differences between PSA-available cases and no- 
PSA-available cases. The likelihood that statistical results are based on true causal 
relationships—and not mere chance—is central to any credible program evaluation. The A2J Lab 
therefore proposed a two-year randomization period. Second, the A2J Lab proposed that each 
criminal case randomized should be followed for two years to ensure that it reached disposition 
and to allow a reasonable time within which to observe arrestees’ behavior after release. 

Given that the A2J Lab envisioned collecting data on two years of completed randomized 
cases, an evaluation based on shorter time periods is unlikely to produce statistically meaningful 
answers. Data from three months of initial appearances in Linn County include 607 study- 
eligible cases, 234 of which have reached final disposition. But as discussed below in Part II.C, 
those completed cases are a non-representative group. The respective numbers in Polk County 
are 6690 and 4266. The upshot is that there is not enough infonnation to produce credible results. 
The A2J Lab therefore does not endorse the numbers reported below. Only more time—and 
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more cases—will allow the generation of useful infonnation about how the PSA has changed (or 
has not changed) pretrial outcomes in Iowa. 

B. Inability to Analyze Outcomes by Arrestee Characteristics 

The problem of insufficient statistical power is enough, in the A2J Lab’s view, to require 
more time to assess the impact of the PSA. There are other significant reasons to wait for more 
data. The national conversation about adopting pretrial risk assessment instruments has included 
their potential benefits and drawbacks. One prevalent concern is the possibility that risk 
assessments induce or reinforce racial disparities in risk prediction and judicial decisions. 

The A2J Lab, in consultation with LJAF and Iowa stakeholders, agreed that providing a 
body of evidence on possible racial disparities would be useful to stakeholders. This objective 
puts further pressure on the statistical power issues described above. When analyzing PSA 
effects by subgroups in the data ( e.g ., outcomes for white versus non-white arrestees) the 
reliability of the statistical test depends on the number of cases within the smallest subgroup. 
Considering that the current case volume is insufficient to measure the impact of the PSA on the 
general population, any subgroup analysis will be still less reliable. 

C. Completed Cases Not Representative of the Whole 

The concerns discussed so far revolve around the need to have a sufficient number of 
cases to draw meaningful conclusions about the impacts of the PSA. There is another limitation 
that calls into question the usefulness of a preliminary report. PSA-eligible criminal cases often 
remain open for a year before reaching disposition. Polk County has held initial appearances on 
over 9000 cases, while Linn County has held just under 1200 study-eligible hearings. Of these 
cases only 5233 and 518 cases in Polk and Linn, respectively, have reached disposition. The A2J 
Lab’s concern for the evaluation is that cases reaching disposition more quickly are different 
from cases that take longer to reach disposition. 

The usefulness of a statistical evaluation rests in part on how closely the cases in the 
study resemble the typical case in the study location. Researchers refer to this issue as one of 
“generalizability.” A PSA study that is not generalizable is not credible. 

Here, the data on which the analysis below rests reflect a particular type of case, one that 
reached disposition within a few months. These cases likely involve less severe charges, 
arrestees with relatively light criminal records, and more straightforward fact patterns. These 
systematic differences mean that the cases on which the analysis below is based are probably not 
representative of all cases in the two counties. 

Without more time, the figures below are not credible. A two-year study with two years 
of follow-up should provide sufficient case volume and time to encompass both the quick- and 
slow-disposition cases needed for a credible analysis. The issues of insufficient statistical power, 
small subgroup sample sizes, and lack of case generalizability hamper our ability to derive 
meaningful conclusions with any level of confidence from the current data. 
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D. Possible Data or Analysis Error 

Finally, the A2J Lab conducted the analysis leading to the present report under conditions 
of haste. Learning the criminal justice and data systems involved in Polk and Linn Counties was 
challenging. Despite the A2J Lab’s best efforts, it cannot rule out an error in data processing or 
analysis at this time. 

III. Pretrial Risk Assessment Instruments and Rando mi zed Control 
Trials 

A. The Basics of Pretrial Release and Money Bail 

i. Procedure 

Initial appearances in criminal cases generally involve the state’s presentation of charges 
and possibly a determination of probable cause for holding the arrestee. 1 Arrestees are usually 
present in person (as in Polk County) or by video feed (as in Linn County). The judge’s 2 primary 
task at the initial appearance is to decide whether the arrestee should be released on his own 
recognizance, released with supervisory conditions, released only if he can post bail (cash or 
surety), or be remanded to the jail (perhaps accomplished by ordering a high bail amount). If the 
judge chooses money bail, the posting of the full value or a surety bond at a percentage of the 
full amount acts like a security deposit to ensure that the arrestee does not fail while awaiting 
disposition. The bail is returned (sometimes after deducting a fee) at case disposition if paid in 
full, but a surety fee is ordinarily not refunded. States ordinarily require judges to consider 
limited criteria when imposing bail; common elements include the judge’s estimation of the 
arrestee’s likelihood of failure to appear for court dates and of the arrestee’s threat to public 
safety ( e.g ., the likelihood of the arrestee committing another criminal act) if released. 3 

A judge may be asked to take a defendant’s ability to pay into consideration, but few 
states cabin judicial discretion regarding the bail amount. 4 Bail can be set at levels that may be 
difficult or impossible for arrestees to pay, even with the presence of a bail bonds industry, 
effectively incarcerating the arrestee until case disposition. 

ii. The Role of Pretrial Risk Assessment Instruments 

Empirical studies have raised questions about the capacity of unguided human decision 
makers to predict future events, including arrestees’ likelihood of failing to appear or committing 
new crimes. 5 Judges making release/detain decisions at initial appearance sometimes lack 


1 This report uses the term “arrestee” to refer to persons formally charged by a prosecutor by the time of initial 
appearance and those whose cases are reviewed based on recommended law enforcement charges. 

2 This report also uses the general term “judge” to refer to the judicial official—alternatively a judge, commissioner, 
or magistrate—presiding over initial appearances. 

3 See Am. Bar Assoc., How Courts Work (2013), available at https://www.americanbar.org/ 
groups/public_education/resources/law_related_education_network/how_courts_work/bail/. 

4 See id. 

5 See Eric Silver & Lynette Chow-Martin, A Multiple Models Approach to Assessing Recidivism Risk: Implications 
for Judicial Decision Making, 29 Crim. Just. & Behav. 538 (2002). 
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complete information about an arrestee, and different judges may weigh available infonnation 
differently. 


These concerns led to the development of algorithms, tools that convert arrestee 
information into risk scores and/or recommendations to help judges assess the likelihood of 
certain behaviors. The hope of these pretrial risk assessment instruments is that they enable 
judges to ground release decisions in more objective and predictive factors than either the charge 
alone or legally fraught characteristics (e.g., race, gender, physical appearance). The instruments 
take a set of inputs, often including a few demographic factors, the charge, and the arrestee’s 
criminal history. Some require information that can only be gleaned from a personal interview. 
These data are then used to produce one or more risk scores, often on a numerical or 
classification scale. 

B. The Public Safety Assessment 

LJAF’s Criminal Justice Initiative “aims to advance community safety and the values of 
equity, fairness, effectiveness, and racial justice.” 6 One LJAF effort in this field has been the 
creation of a pretrial risk assessment instrument, one that would not require an arrestee interview 
to implement. The result is the PSA, which used 750,000 cases from approximately 300 
jurisdictions across the United States to determine the risk factors that best predict an arrestee’s 
likelihood of failing to appear in court or being arrested on a new criminal charge. The PSA 
relies on an arrestee’s criminal history, age, and charge in compiling scores associated with risk 
of new (and new violent) criminal activity and failure to appear. LJAF has collaborated with 
academics and other researchers to evaluate the PSA in adopting locations. 

C. Randomized Control Trials 

Determining whether a change in policy caused observed outcomes (such as reduced 
incarceration, failure to appear, new criminal activity, or new violent criminal activity) is a 
difficult task. LJAF has engaged the A2J Lab to design, implement, and analyze two randomized 
control trial (“RCT”) evaluations of the PSA. RCTs are the gold standard for determining 
whether a policy or program caused outcomes. By way of analogy, the FDA currently requires 
RCT evaluations of new drugs before they can be marketed to the public. RCTs rely on the 
random selection of units (e.g., criminal cases, arrestees) to create a “control group,” which 
usually follow the status quo, and a “treated group” that is exposed to the policy or program. The 
PSA RCTs create control groups in which the PSA is not available to a judge at initial 
appearances and treated groups in which the PSA is available. After the field experiment closes, 
researchers will compare outcomes between control and treated groups for evidence of any 
statistically significant difference. 

The RCT will provide credible information as to PSA’s effects. So-called “before-and- 
after” comparisons cannot provide credible information in the present context because they 
cannot account for factors that operate alongside a policy or program, such as changes in 
criminal justice policy that occurred at the same time as PSA adoption. Governments often adopt 
other, related policies at the same time they launch the one being evaluated. Populations also 


6 The Laura and John Arnold Foundation, Criminal Justice , http://www.arnoldfoundation.org/initiative/criminal- 
justice/. 
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change. Community demographics and economic conditions fluctuate over time, and, when they 
do at the same time as a policy’s introduction, it becomes difficult or impossible to attribute 
changes in observed outcomes to the policy change. When properly designed and implemented 
with sufficient case volume, RCTs allow researchers to isolate the effects that the policy alone 
has on study participants, in this case arrestees. 

IV. Descriptions of the PSA RCTs in Iowa 

A. Polk County 

The Polk County PSA RCT launched when its initial appearance court started using the 
PSA in January of 2018. The study randomizes PSA-eligible, in-custody appearances (rather 
than arrestees); therefore, the unit of analysis is a court case, not a person. The treated group 
consists of all initial appearances where the associated booking number ends in an even number; 
control cases are bookings with odd-numbered identifiers. The Fifth District Department of 
Correctional Services provides the PSA report and recommendation to initial appearance judges 
in treated cases and not in control ones. The primary outcomes of interest include: (1) the 
number of days arrestees spend incarcerated before case disposition; (2) failures to appear 
(“FTA”); (3) new criminal activity (“NCA”); and (4) new violent criminal activity (“NVCA”). 

B. Linn County 

The Linn County PSA RCT launched when its initial appearance courts started using the 
PSA in July of 2018. The study randomizes PSA-eligible, in-custody arrestees, rather than cases; 
therefore, the unit of analysis is a person, not a case. The treated group consists of all initial 
appearances where the arrestee’s unique Iowa Corrections Offender Network (“ICON”) 
identification number ends in an even number; control units are persons with odd-numbered 
ICON identifiers. The Sixth District Department of Correctional Services provides the PSA 
report and recommendation to initial appearance judges in treated cases and not in control ones. 
The primary outcomes of interest include: (1) the number of days arrestees spend incarcerated 
before case disposition; (2) failures to appear (“FTA”); (3) new criminal activity (“NCA”); and 
(4) new violent criminal activity (“NVCA”). 

C. Outcome Variable Descriptions 

All outcome information for the Polk and Linn County Studies is available through Iowa 
Courts Online (“ICIS”), the Polk County Jail’s database, the Linn County Jail’s database, and a 
statewide hub known as the Justice Data Warehouse. The relevant outcome variables and 
descriptions are: 

1. FTA: A 0-1 variable capturing whether the arrestee missed any court date before 
disposition of the case. 

2. NCA: A 0-1 variable, capturing whether the arrestee was arrested during the pretrial 
period for an offense subject to a possible incarceration sentence. This variable is 
computed once for any NCA committed during the pretrial period (from initial 
appearance to case disposition), and, in the full study (but not in the present report), again 
for any NCA arrest between the initial appearance and two years after the initial 
appearance. 
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3. NVCA: A 0-1 variable capturing whether the arrestee was arrested during the pretrial 
period for a violent offense subject to a possible incarceration sentence. This variable is 
computed once for any NVCA committed during the pretrial period (from initial 
appearance to case disposition), and, in the full study (but not in the present report), again 
for any NVCA arrest between the initial appearance and two years after the initial 
appearance. 

4. Length of pretrial incarceration: The number of days the arrestee spent incarcerated 
between the initial appearance and case disposition. 

5. Length of the pretrial period: The number of days between the initial appearance and case 
disposition. 

6. Conviction: A 0-1 variable capturing whether the arrestee was convicted of any charge 
and as a 0-1 variable addressing the lead charge, i.e., whether the arrestee was convicted 
of the charge carrying the longest possible term of incarceration. 

7. Sentence: A 0-1 variable capturing whether the arrestee was sentenced to any term of 
incarceration and as the number of days the arrestee was ordered to serve, if any. 

V. Analysis and Discussion 

The final Part of the report is limited to the following two analyses: (1) a summary of 
collected data, including a description of the cases in the current dataset, the distributions of 
treated and control cases by disposition status, and the distributions of release/detain decisions by 
experimental treatment and judicial concurrence; and (2) comparisons between the control (no- 
PSA-provided) and treated (PSA-provided) groups for the primary outcomes of FTA, NCA, 
NVCA and time spent incarcerated before disposition. 

A. Data Validation 

As described in Part IV, the A2J Lab used several databases to track individuals and 
cases. To minimize errors, anomalies, or incorrect attributions, the A2J Lab established processes 
to validate data across the data sources. These processes included use of‘independent dual links” 
between data sources and random sampling of cases for manual verification. Independent dual 
links refers to the method by which the A2J Lab is able to connect a jail booking to a court case 
and a court case to an FTA, NCA, NVCA, or disposition incident, all across separate state- and 
county-level databases. The A2J flagged any cases that did not match across databases. Data 
managers within the Iowa court system and the county jail systems re-verified these cases and 
returned them to the A2J Lab. Once all the cases were validated and matched, a handful were 
randomly selected for manual verification. These procedures help ensure that the data collected 
and analyzed for this report accurately represent outcomes in Polk and Linn Counties. 

B. Descriptive Statistics 

Figures 1 and 2 below detail the current counts and distributions of study-eligible cases 
across the two evaluation sites. The cases are also divided according to their disposition status. In 
both Polk and Linn Counties, roughly half of the eligible cases have reached disposition (slightly 
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more in Polk and slightly less in Linn). Full data analysis can begin only after a case has reached 
disposition, and for some outcomes of interest the timeframe is even longer. 

Figures 1 and 2 also divide cases by treatment condition, i.e., whether a PSA printout was 
received by the district associate judge. The number of treated (PSA-provided) to control (no- 
PSA-provided) cases are roughly equivalent across counties and disposition status, indicating 
that the randomization scheme in place is functioning as intended. Lastly, the number of study- 
eligible units in Linn County does not reflect the number of cases included the data. The reason 
is that the study in Linn County is focused on (and applies the PSA treatment by) individuals 
rather than cases. 


Figure 1 


Number and Type of Cases Across 
Trial Status in Polk County 
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Figure 2 


Number and Type of Cases Across 
Trial Status in Linn County 
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Figures 3 and 4 detail the distribution of cases across categories of release/detain 
decisions made by judges. A key consideration in evaluating the PSA is understanding whether 
judges actually utilize the information provided, since the PSA is entirely non-binding in nature. 
Counts are reported for each possible release/detain decision made or release/detain 
recommendation ranging from own-recognizance release to release not recommended, as well as 
by the judges’ concurrence with the PSA recommendation. Cases in which arrestees pleaded 
guilty at initial appearance or in which judges issued orders inconsistent with the PSA 
release/detain options were not included in Figure 2. In Polk County, judicial decisions tend to 
be unifonnly spread over the range of release options, with the exception of PM4 release, which 
is rare. In Linn County, judges tended to be more likely to release on the arrestee’s own 
recognizance than the PSA suggested, although the PSA also recommended own-recognizance 
release more often than any other outcome. 

Figures 3 and 4 also show the distributions of judicial release outcomes when the judge 
concurred (or did not concur) with the PSA recommendation. In Polk County, concurrence 
occurred in 58% of cases and concentrated primarily on the endpoints of the release spectrum 
scale: own-recognizance release or release not recommended. Disagreement between PSA 
recommendations and judicial decisions tended to occur over the level of conditional release. In 
Linn County, judges concurred with the PSA recommendations in 83% of cases, leaving too few 
disagreements to provide meaningful analysis. 
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Figure 3 


Judicial Release Decision Outcomes in Polk County 
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Figure 4 


Judicial Release Decision Outcomes in Linn County 
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C. Outcomes and Analysis 

For each of the primary outcomes, comparisons are made between the average rate (or 
length of time) for the treated (PSA-provided) and the control (no-PSA-provided) groups. These 
comparisons allow researchers to estimate the causal effect that the treatment policy has on the 
outcomes of interest. The analysis uses the total number of study-eligible cases, partly defined as 
having reached disposition, not the overall number of cases during the study period. Below, two 
distinct statistical metrics are reported for each outcome. The first represents whether we can 
confidently say that any observed difference likely reflects an underlying true difference, as 
opposed to random chance. This figure is referred to as the p-value. The reported /^-values do not 
correct for multiple hypothesis testing on the same data, so the true /^-values are likely to be 
higher once the inherent connection among outcome variables is taken into account. The second 
metric reported is a measurement of the policy’s effect size, what statisticians call a confidence 
interval. The confidence interval here tells us the range in which the PSA effect would fall 95% 
of the time if we were to repeat the same experiment many times. 7 

Figures 5 and 6 show the distribution of NCA rates across treated groups and by county. 
Of the roughly 2100 cases in Polk County in which a PSA report was available, about 20.5% had 
at least one recorded NCA prior to their case disposition. Compare this to the 17.9% of cases 
where no PSA report was available. If the data analyzed were representative of all cases in Polk 


7 For NCA, NVCA, and FTA, /j- values measuring the statistical significance of the difference between the control 
and treated group are calculated using a permutation, or Fisher’s exact, test. See Lynne M. Connelly, Fisher’s Exact 
Test, 25 MEDSURG Nursing 58 (2016). For total time served, the Wilcoxon rank sum statistic is used. See Frank 
Wilcoxon, Individual Comparisons by Ranking Methods, 1 Biometrics Bull. 80 (1945). 
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County, this difference would be observed with a /?-value of 0.03. But, as noted above, the 
analyzed cases are not representative of all cases in Polk County, and there has been no 
correction for multiple testing. The estimated effect size of the PSA exists on an interval from 
0.2% to 5%. This range—between roughly a fifth of a percent increase and a 5% increase in the 
NCA rate—is fairly large and reflective of the low power and high uncertainty from which the 
study suffers. Note, however, that the 20.5% NCA rate is below the 2016 baseline of 22.8%. It is 
likely that over time, however, more individuals will be arrested for NCA, bringing both control 
and treated group rates closer to the prior baseline. The PSA effects on NCA point in the 
opposite direction for Linn County with a/> value of 0.435. The associated confidence interval 
for the effect size ranges from -5.6% to 2.5%. 


Figure 5 


NCA Rates Across Treatment Groups in Polk County 
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Figure 6 


NCA Rates Across Treatment Groups in Linn County 
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Figures 7 and 8 perform the same calculations but for the subset of NCA that are 
classified as violent crimes, or NVCA. NVCA is a considerably rarer occurrence, with only 
about 1% of arrestees in Polk County (52 total) being arrested for at least one NVCA charge. The 
interval for the effect size of the PSA on NVCA occurrences is between a 0.2 % and 1.6% 
increase, with a /?-value of .016; again, the data are not representative of all Polk County cases, 
and there has been no correction for multiple testing. The corresponding interval in Linn County 
is -2.5% to 0.8%,/? = 1; again, the data are not representative of all cases in Linn County (as 
evidenced by the fact that the data appear to show no NVCA incidence in the Linn County 
treated group). 
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Figure 7 


NVCA Rates Across Treatment Groups in Polk County 

p-value= 0.016 ; 57 Total Instances 
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Figure 8 


NVCA Rates Across Treatment Groups in Linn County 

p-value= 1 ; 1 Total Instances 
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Figures 9 and 10 show differences between the treated and control groups for FTA rates. 
In Polk and Linn Counties, the corresponding /^-values are 0.298 and 0.487, respectively. In Linn 
County, there was no recorded instance of FTA when the PSA was not provided at initial 
appearance (and only 1 when the PSA was provided), reemphasizing that the cases available for 
analysis are not representative of the whole. The confidence intervals for the effect size of the 
PSA on FTA rates is between -0.5% and 1.8 % in Polk County and between -0.9% and 2.6% in 
Linn County. 


Figure 9 


FTA Rates Across Treatment Groups in Polk County 

p-value= 0.298 ; 162 Total Instances 
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Figure 10 


FTA Rates Across Treatment Groups in Linn County 

p-value= 0.487 ; 1 Total Instances 
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The analysis in Figures 11 and 12 differs from the others in that it shows an average 
length of time, in days, as opposed to an incident rate of a specific post-release outcome. 
Arrestees may be incarcerated before disposition for different lengths of time due to any of the 
following reasons, among others: not being released, inability to post bail, or being arrested on a 
new criminal charge while released. The pre-disposition incarceration measurement focuses on 
the total number of days an individual was in the county jail while their case was active. Since 
active incarceration makes committing NCA, NVCA, or FTA impossible, understanding the 
relative lengths of incarceration is necessary for interpreting the other outcomes. In Polk County, 
arrestees in the PSA treated group served 2.2 fewer days on average than their non-PSA 
counterparts, a decrease of 7.8%, with ap-value of 0.001, and a confidence interval of -4.5 days 
to -0.1 days. Again, these cases are not representative of the whole, and there has not been a 
correction for multiple testing. In Linn County, the results were in the opposite direction, with 
PSA arrestees serving on average 2.8 days more in pretrial detention, but with a/?-value of 0.279 
and a confidence interval between -2.8 days and 8.3 days. These cases likely have incarceration 
times shorter than those in the whole dataset, given the fact they were among the first to reach 
disposition. 
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Figure 11 


Average Time Served Across Treatment Groups in Polk County 

p-value= .001 ; 112049 Total Days Served 




Figure 12 


Average Time Served Across Treatment Groups in Linn County 

p-value= 0.279 ; 1947 Total Days Served 
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VI. Addendum 


The Access to Justice Lab produced its original report under conditions of unusual haste. 
The problem of rapid turnaround was compounded when the data the A2J Lab received to 
generate the report was fonnatted differently from the test data the A2J Lab had received several 
weeks earlier. These formatting differences resulted in lost hours spent reprogramming the 
analysis software. As a result, the A2J Lab could not complete data integrity checks that it would 
ordinarily have implemented before the report was issued; the report included a warning 
regarding data quality and institutional comprehension of the data in its initial report. Subsequent 
to issuing the initial report, the A2J Lab continued verifying the data and performing integrity 
checks, and it discovered four issues, detailed below. 

None of these issues, or the corrections implemented, alter the central conclusion of the 
A2J Lab’s report: there are insufficient data at this time to conclude that use of the PSA in either 
Polk or Linn County affects the four key outcomes of (1) days spent incarcerated before case 
disposition; (2) new criminal activity; (3) new violent criminal activity; (4) or failure to appear. 
Corrections resulted in changes to the figures and significance levels in the original report, but 
not to the overall conclusions. Only additional time will allow credible inferences as to the 
effects of the PSA on key outcomes. 

This addendum details the remaining data issues, the relative number of cases affected by 
such issues, and what efforts have been made (and will be done) to address them. The data relied 
upon to produce the original report implicate four separate issues: (1) PSA cases that could not 
be matched to the Iowa Courts Online Database; (2) the presence of individuals who pleaded 
guilty during their initial appearance; (3) the presence of individuals who posted bail prior to 
their initial appearances; and (4) multiple dates associated with a single case ID. Each of these 
problems, their relative size, and what attempts have been and will be taken to address them is 
detailed briefly below. 

1) PSA eligible cases that could not be matched to Iowa’s state court database 

During the period for which cases were collected (January 16, 2018 - October 3, 2018), 
Polk County produced roughly 9000 unique cases for which a PSA was generated (across both 
control and treated groups). One should be able to locate each of these 9000 cases, along with all 
other cases associated with the arrestee for which a PSA was generated, to data stored within 
Iowa Courts Online/the Justice Data Warehouse. Only 7700 of the 9000 cases were matched; 
roughly 1300 cases did not produce a match and thus were excluded from the analysis due to 
incomplete information. A secondary round of matching was able to recover roughly 1200 of the 
missing cases, with 100 having corrupted/non-standardized case IDs. 

2) Individuals pleading guilty during initial appearance 

An individual booked on a PSA-eligible charge can plead guilty during their initial 
appearance. Pleading guilty at this stage results in the disposition date and initial appearance date 
occurring on the same day. Arrestees who plead guilty on the day of their initial appearance have 
a total pretrial period of zero days. Analysis of the PSA’s impact relies on observing individuals 
released on bail during this pretrial phase. By including individuals who never had a chance to 
commit an NCA or FTA during their pretrial period, the instances of zero counts (zero NCA, 
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zero NVCA, zero FT A, and zero time spent incarcerated) increase at the expense of providing 
meaningful information. The A2J Lab addressed this problem by removing all individuals with a 
pretrial period of zero days, dropping roughly 1100 cases from the analysis. 

3) Individuals bonding out prior to initial appearance 

Iowa allows individuals booked on certain charges (many of which are PSA-eligible) to 
post a preset amount of bail, set by state statute, prior to an initial appearance. In these instances, 
an individual becomes ineligible for the study, since a judge had no opportunity to set release 
conditions (with or without PSA input). These cases can be identified directly by observing the 
outcome of an initial appearance hearing. But this information is, at this point, only collected for 
individuals in the treated group. There are currently 54 instances of bonding out prior to initial 
appearance within the treated group data. Assuming roughly the same number of instances 
occurred for the control group, there are potentially around 100 cases within the data subject to 
ineligibility on this ground. The A2J Lab continues to work with Iowa data managers to address 
this problem. 

4) Multiple dates associated with a single case ID 

Initial data checks indicated that case IDs could be associated with multiple incident 
dates within Iowa Courts Online. In the original report’s data, incident dates associated with a 
case ID are always reported with a single date. But multiple dates could be associated with a 
single case, specifically when a contempt charge in the current case was appended to an older, 
inactive case. There are nine recorded instances of this occurring in the data. Additional random 
sampling has not identified other cases where multiple incident dates on a single case ID have 
produced a misleading NCA count. The A2J Lab continues to work with Iowa data managers to 
address this problem. 
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